在本文中，我们通过变换量化压缩卷积神经网络（CNN）权重。以前的CNN量化技术倾向于忽略权重和激活的联合统计，以给定的量化比特率产生次优CNN性能，或者在训练期间考虑其关节统计，并且不促进已经训练的CNN模型的有效压缩。我们最佳地转换（去相关）并使用速率失真框架来量化训练后的权重，以改善任何给定的量化比特率的压缩。变换量化在单个框架中统一量化和维度减少（去相关性）技术，以促进CNN的低比特率压缩和变换域中的有效推断。我们首先介绍CNN量化的速率和失真理论，并将最佳量化呈现为速率失真优化问题。然后，我们表明，通过在本文中获得的最佳端到端学习变换（ELT），可以使用最佳位深度分配来解决此问题。实验表明，变换量化在雷则和非烫伤量化方案中推进了CNN压缩中的技术状态。特别是，我们发现使用再培训的转换量化能够压缩CNN模型，例如AlexNet，Reset和DenSenet，以非常低的比特率（1-2比特）。
translated by 谷歌翻译
Large language models (LLMs) have shown impressive results across a variety of tasks while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial for both system developers and users in this setting. We propose and study Attributed QA as a key first step in the development of attributed LLMs. We develop a reproducable evaluation framework for the task, using human annotations as a gold standard and a correlated automatic metric that we show is suitable for development settings. We describe and benchmark a broad set of architectures for the task. Our contributions give some concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third key question (How to build LLMs with attribution?).
translated by 谷歌翻译
Recent years have seen rapid progress at the intersection between causality and machine learning. Motivated by scientific applications involving high-dimensional data, in particular in biomedicine, we propose a deep neural architecture for learning causal relationships between variables from a combination of empirical data and prior causal knowledge. We combine convolutional and graph neural networks within a causal risk framework to provide a flexible and scalable approach. Empirical results include linear and nonlinear simulations (where the underlying causal structures are known and can be directly compared against), as well as a real biological example where the models are applied to high-dimensional molecular data and their output compared against entirely unseen validation experiments. These results demonstrate the feasibility of using deep learning approaches to learn causal networks in large-scale problems spanning thousands of variables.
translated by 谷歌翻译
Modern machine learning models are often constructed taking into account multiple objectives, e.g., to minimize inference time while also maximizing accuracy. Multi-objective hyperparameter optimization (MHPO) algorithms return such candidate models and the approximation of the Pareto front is used to assess their performance. However, when estimating generalization performance of an approximation of a Pareto front found on a validation set by computing the performance of the individual models on the test set, models might no longer be Pareto-optimal. This makes it unclear how to measure performance. To resolve this, we provide a novel evaluation protocol that allows measuring the generalization performance of MHPO methods and to study its capabilities for comparing two optimization experiments.
translated by 谷歌翻译
The ability to capture detailed interactions among individuals in a social group is foundational to our study of animal behavior and neuroscience. Recent advances in deep learning and computer vision are driving rapid progress in methods that can record the actions and interactions of multiple individuals simultaneously. Many social species, such as birds, however, live deeply embedded in a three-dimensional world. This world introduces additional perceptual challenges such as occlusions, orientation-dependent appearance, large variation in apparent size, and poor sensor coverage for 3D reconstruction, that are not encountered by applications studying animals that move and interact only on 2D planes. Here we introduce a system for studying the behavioral dynamics of a group of songbirds as they move throughout a 3D aviary. We study the complexities that arise when tracking a group of closely interacting animals in three dimensions and introduce a novel dataset for evaluating multi-view trackers. Finally, we analyze captured ethogram data and demonstrate that social context affects the distribution of sequential interactions between birds in the aviary.
translated by 谷歌翻译
2022-11-11
We introduce KPI-Check, a novel system that automatically identifies and cross-checks semantically equivalent key performance indicators (KPIs), e.g. "revenue" or "total costs", in real-world German financial reports. It combines a financial named entity and relation extraction module with a BERT-based filtering and text pair classification component to extract KPIs from unstructured sentences before linking them to synonymous occurrences in the balance sheet and profit & loss statement. The tool achieves a high matching performance of $73.00$% micro F$_1$ on a hold out test set and is currently being deployed for a globally operating major auditing firm to assist the auditing procedure of financial statements.
translated by 谷歌翻译
我们介绍了第一个机器学习引力波搜索模拟数据挑战（MLGWSC-1）的结果。在这一挑战中，参与的小组必须从二进制黑洞合并中识别出复杂性和持续时间逐渐嵌入在逐渐更现实的噪声中的引力波信号。 4个提供的数据集中的决赛包含O3A观察的真实噪声，并发出了20秒的持续时间，其中包含进动效应和高阶模式。我们介绍了在提交前从参与者未知的1个月的测试数据中得出的6个输入算法的平均灵敏度距离和运行时。其中4个是机器学习算法。我们发现，最好的基于机器学习的算法能够以每月1个的错误警报率（FAR）的速度（FAR）实现基于匹配过滤的生产分析的敏感距离的95％。相反，对于真实的噪音，领先的机器学习搜索获得了70％。为了更高的范围，敏感距离缩小的差异缩小到某些数据集上选择机器学习提交的范围$ \ geq 200 $以优于传统搜索算法的程度。我们的结果表明，当前的机器学习搜索算法可能已经在有限的参数区域中对某些生产设置有用。为了改善最新的技术，机器学习算法需要降低他们能够检测信号并将其有效性扩展到参数空间区域的虚假警报率，在这些区域中，建模的搜索在计算上很昂贵。根据我们的发现，我们汇编了我们认为，将机器学习搜索提升到重力波信号检测中的宝贵工具，我们认为这是最重要的研究领域。
translated by 谷歌翻译
对比度学习是视觉表示学习最成功的方法之一，可以通过在学习的表示上共同执行聚类来进一步提高其性能。但是，现有的联合聚类和对比度学习的方法在长尾数据分布上表现不佳，因为多数班级压倒了少数群体的损失，从而阻止了学习有意义的表示形式。由此激励，我们通过适应偏见的对比损失，以避免群集中的少数群体类别的不平衡数据集来开发一种新颖的联合聚类和对比度学习框架。我们表明，我们提出的修改后的对比损失和分歧聚类损失可改善多个数据集和学习任务的性能。源代码可从https://anonymon.4open.science/r/ssl-debiased-clustering获得
translated by 谷歌翻译
声词嵌入（AWES）的模型学会将可变长度的口语段映射到固定差异矢量表示上，以便在嵌入空间附近预计，同一单词的不同声学示例。除了他们的语音技术应用外，AWE模型还显示出可以预测各种听觉词汇处理任务的人类绩效。当前的敬畏模型基于神经网络，并以自下而上的方法进行了培训，该方法集成了声音提示，以构建给定声或符号监督信号的单词表示。因此，这些模型在学习过程中不会利用或捕获高级词汇知识。 ％并捕获有关单词形式的低级信息。在本文中，我们提出了一个多任务学习模型，该模型将自上而下的词汇知识纳入了敬畏的训练程序中。我们的模型学习了声学输入和词汇表示之间的映射，该表示除了基于自下而上的表单监督外，还编码了高级信息，例如单词语义。我们尝试三种语言，并证明合并词汇知识可以改善嵌入空间的可区分性，并鼓励模型更好地分开词汇类别。
translated by 谷歌翻译
抗原加工途径的硅硅建模准确性对于实现个性化表位疫苗设计至关重要。这种途径的一个重要步骤是，蛋白酶体将疫苗降解为较小的肽，其中一些将由MHC复合物呈现给T细胞。虽然最近预测MHC肽的表现引起了很多关注，但鉴于高通量质谱的MHC连接学的最新进展，蛋白酶体裂解预测仍然是一个相对未探索的区域。此外，由于这种实验技术不允许识别无法分裂的区域，因此最新的预测因子会在训练时会产生合成的负样本并将其视为真正的负面样本，即使其中一些实际上可能是肯定的。因此，在这项工作中，我们提出了一个新的预测指标，该预测因素通过扩展的数据集和稳固的未标记学习理论基础进行了培训，从而实现了蛋白酶体裂解预测的新最新。改进的预测能力反过来又可以使更精确的疫苗开发提高基于表位的疫苗的功效。可以在https://github.com/schubertlab/proteasomal-cleavage-puupl上获得代码和预估计的模型。
translated by 谷歌翻译